Skip to content

Reclaim idle connections across routes in RouteSegmentedConnPool - #722

Merged
arturobernalg merged 1 commit into
apache:masterfrom
arturobernalg:offlock-cross-route-reclaim
Oct 26, 2026
Merged

arturobernalg merged 1 commit into
apache:masterfrom
arturobernalg:offlock-cross-route-reclaim

Conversation

@arturobernalg

@arturobernalg arturobernalg commented Oct 3, 2026 •

Copy link
Copy Markdown
Member

RouteSegmentedConnPool stopped allocating when the global limit was reached, even if another route held an idle reusable entry, and its hot path paid for two shared structures: the executor hand-off of every graceful close and the single deque of idle entries of each route.

Three commits:

  1. Reclaim idle entries across routes at max capacity. When maxTotal is reached, an idle entry of another route is reclaimed and its global slot transferred to the requesting route. The hot path stays route-local; pending routes are retried when capacity appears.
  2. Lock-free disposal and drain hand-offs. Graceful closes go through a bounded lock-free queue served by on-demand daemon threads: a caller wakes a worker only when none is awake, idle workers linger in short timed parks, and workers bring in more threads themselves when their closes are slow or the backlog grows. close() never waits on the backlog. The round-robin drain runs on a single on-demand thread signalled with a CAS. Segments are retired with a negative allocation marker before leaving the map, which fixes a global slot leak when an allocation raced the removal of a segment.
  3. Per-thread idle entries. Under LIFO the idle entries of a route are kept in independent Treiber stacks, one per stripe, with the heads on separate cache lines; a thread pushes to and pops from its own stripe and steals from another only when it is empty. Reuse is most recent per thread rather than per route, and eviction takes the least recently released entry of one stripe. FIFO keeps the deque.

Benchmarks

JMH 1.37, JDK 17, 16 cores, -f 2 -wi 3 -w 1 -i 5 -r 1, each suite run alone. RouteSegmentedConnPoolBenchmark has no I/O and 8 threads unless stated; RoutePoolsJmh drives real sockets against a local HTTP/1.1 server with 50 threads, maxTotal=100, 5 per route, and reports completed requests.

Benchmark OFFLOCK STRICT LAX
hotRouteUncontended (lease hit + release, 8 threads), ops/s 69.8 M ± 2.2 M 3.64 M 3.55 M
hotRouteUncontended, 1 thread, ops/s 15.7 M 6.2 M 10.1 M
coldRouteAtFullPoolWithIdle (reclaim at maxTotal), ops/s 3.46 M 1.58 M n/a
coldRouteProgressUnderSameRouteBacklog, mean latency 0.34 µs 0.19 µs n/a
hotRoute / hotRouteAtFullPool (1 connection, 8 threads), ops/s 229 k / 231 k 231 k / 229 k 222 k / 221 k
RoutePoolsJmh, 50 routes, 200 ms graceful close, 0 % / 20 % discards, req/s 2410 / 2551 1 / 3 2320 / 940
RoutePoolsJmh, 10 routes, 200 ms graceful close, 0 % / 20 % discards, req/s 1067 / 1262 1055 / 507 1065 / 537
RoutePoolsJmh, 50 routes, instant close, 20 % discards, req/s 2784 2493 2506

Before these changes OFFLOCK measured 3.6 M ops/s on hotRouteUncontended, 1.9 M on coldRouteAtFullPoolWithIdle and 1.4 µs on the backlog case, and the socket benchmark with slow closes did not complete. hotRoute with a single connection and the instant-close socket rows are bound by thread hand-off latency and by the server respectively, so all three pools measure the same there. STRICT closes connections inline, which is why it leads the backlog latency with no-op closes and collapses with real ones.

httpcore5: 2007 tests, Checkstyle clean.

@arturobernalg
arturobernalg marked this pull request as draft October 3, 2026 15:45
@arturobernalg
arturobernalg force-pushed the offlock-cross-route-reclaim branch from bc8c1b4 to 715f22a Compare October 3, 2026 17:27
@arturobernalg
arturobernalg marked this pull request as ready for review October 4, 2026 04:40
@arturobernalg
arturobernalg force-pushed the offlock-cross-route-reclaim branch 2 times, most recently from 9159a06 to bf93e5e Compare October 4, 2026 15:55
@arturobernalg
arturobernalg force-pushed the offlock-cross-route-reclaim branch from fb0e2f9 to b4b5977 Compare October 13, 2026 12:33
@arturobernalg
arturobernalg force-pushed the offlock-cross-route-reclaim branch from c8a2cd3 to 348f961 Compare October 13, 2026 12:57
@arturobernalg
arturobernalg merged commit 3d3b9d2 into apache:master Oct 26, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant